The use of data communications networks has increased dramatically over the past ten years, driven by both technical progress, such as ease of use and economical access, and by necessity, for example the global business environment. Generally this progress has been good due to gains in efficiency and timeliness of information, resulting in more accurate data and hence better decision making. But as the use of networks increases by both private and public entities, so does the dependency upon the data carried over the networks. Attendant with the increased dependency is an increased vulnerability to attack by persons bent on mischief, for whatever reason.
While there has been an increase in the positive effects of wide-spread network use, so too there has been an increase in the negative effects. Specifically, the incidences of attacks by intruders, or so-called hackers, has seen dramatic increases, causing major network crashes such as those seen by Yahoo and eBay in late 1999. One such variety of attack, a Denial Of Service [DOS], also known generally as a “flood,” effectively swamps the target network with so many requests for service that no response is possible, thereby debilitating a family of routers and possibly, in theory at least, an entire network. This is an extremely difficult attack to prevent for a variety of reasons, thus there exists a need to mitigate.
But mitigation is difficult because modern network architectures are susceptible to hostile attack. Reasons for this susceptibility are, among others, the global nature of the threat including both recreational and terrorist attackers, the multi-platform/multi-protocol nature of the networks involved, and the constant change taking place in the network community. As noted at the CERT® Distributed-Systems Intruder Workshop, “Intruders are actively developing distributed tools . . . ” making attacks easier, in part “ . . . because of the large number of machines ‘available for public use.’” [Results of the Distributed-Systems Intruder Tools Workshop, Pittsburgh, Pa., Nov. 2–4 1999, p. 3]. Public use machines could be, for example, those located in libraries or academic computer labs and accessible to the general public. Such machines can be made the unwitting accomplices in a DOS attack, yielding a multiplier effect focused on the target server, router or network.
Attacks are made easier as well because it is difficult to separate legitimate traffic patterns from hostile patterns. Generally, network traffic may be separated into three broad categories: known good, known bad and questionable. Tools are prevalent which allow the determination of which category a specific data stream falls into, but each interferes with the data flow to one extent or another. Where the data flows are very high volume, as is the case in the emerging fiber optic network data pipes, this interference could become a burden on the system performance. Some current methods include serial data stream filtering, encryption, data stream sampling and data stream throttling.
By far the most widely used current method is serial filtering where all ingress data is sent through the filter and checked for known bad patterns. Encryption uses a key that is passed from client to server in order to validate the data. Sampling techniques look at random data streams over varying periods of time to recognize normal patterns. Throttling techniques involve reducing the amount of traffic allowed across the network in response to abnormal volume. Each of these methods, however, suffer from deleterious effects on the performance of the network ranging from mild to severe, depending upon the level of validation sought.
Further complicating the security problem is, that although intruder methods are well understood by those of skill in the art, an attack is difficult to detect until well after it is under way. Add to this the forging of IP addresses, or spoofing as it is called, the category of a particular data stream can be extremely difficult to determine in real time. All of the above mentioned methods suffer from this inability to rapidly detect an attack versus a legitimate variation in a data stream. While filtering methods may guarantee the validity of all data in a stream, it does so by severely limiting the amount of traffic that may pass. The same may be said about encryption and throttling to one extent or another. Sampling methods suffer from the inability to monitor the entire IP address space of a network in real time, thereby potentially missing the onset of an attack.
The present invention significantly advances the art through the ability to detect and react to certain types of attacks while they are commencing and to do so in the entire address space of a network. These and other advantages of the present invention are discussed in detail below in conjunction with the figures attached.